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Abstract 

In  today’s  Air  Force,  we  are  constantly  faced  with  meeting  the  mission  under  limited 
budget  and  equipment  constraints.  Any  tools  that  make  completing  the  mission  easier  are 
highly  sought  after,  and  rightfully  so.  The  Air  Mobility  Planner’s  Calculator 
(AMPCALC)  is  one  such  tool.  It  gives  the  mobility  planner  flexibility  in  planning  by 
allowing  several  different  situations  to  be  modeled.  This  paper  recommends 
improvements  to  the  model  and  provides  the  verification  and  validation  of  the  model  in 
an  attempt  to  prove  that  the  model  is  a  useful  tool  for  Air  Mobility  Command  and  the 
United  States  Air  Force. 
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THE  AIR  MOBILITY  PLANNER’S  CALCULATOR:  IMPROVEMENTS, 


VERILI CATION,  AND  VALIDATION 


I.  Introduction 

Air  Mobility  Command’s  primary  role  in  today’s  Air  Force  is  Rapid  Global 
Mobility,  followed  closely  by  Agile  Combat  Support.  Rapid  Global  Mobility  means  that 
we  are  able  to  get  people  and  equipment  anywhere  on  the  globe  quickly.  Agile  Combat 
Support  means  that  we  do  it  efficiently.  In  today’s  conflicts,  we  do  not  have  the  money 
or  the  resources  to  throw  at  a  problem  until  it  goes  away.  There  is  no  longer  room  for 
waste.  Therefore,  it  falls  to  the  air  mobility  planner  to  look  at  the  demands  for  airlift, 
prioritize  them,  and  put  airplanes  against  them  in  order  that  people  and  equipment  are 
moved  in  a  timely  and  efficient  manner.  Considering  that  this  occurs  on  a  global  scale, 
the  task  of  the  planner  is  immense  and  constantly  changing.  One  tool  that  will  help  the 
planner  is  the  newly  developed  Air  Mobility  Planner’s  Calculator  (AMPCALC). 

AMPCALC  is  a  deterministic  model  that  quickly  resolves  issues  such  as  which 
airframe  to  use,  flight  times,  loading  and  unloading  times,  ramp  space,  and  virtually 
everything  else  having  to  do  with  the  air  mobility  process.  The  model  was  developed  at 
the  Air  Force  Institute  of  Technology  in  2002  (Pektas,  2002).  This  research  continues 
that  work  by  making  improvements  to  the  model  and  performing  verification  and 
validation  of  the  model.  The  ultimate  goal  is  to  provide  AMC  with  a  tool  that  can  be 
used  regularly  to  quickly  and  accurately  give  air  mobility  planners  an  idea  about  how  an 
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air  movement  should  perform  and  to  give  them  a  basic  understanding  of  the  levels  and 
timing  of  throughput. 

The  model  developed  by  Pektas  was  well  done,  but  feedback  from  AMC  and 
potential  users  have  yielded  areas  for  improvement.  One  improvement  is  to  integrate  the 
four  airlift  cycles  that  are  currently  separate.  They  were  initially  designed  separately  in 
order  to  make  comparisons  of  different  routes  quickly.  However,  AMC  has  expressed  an 
interest  in  having  the  cycles  share  resources  in  order  to  provide  a  more  realistic  picture  of 
an  airlift  movement.  In  many  cases,  passengers  and  cargo  will  originate  at  different 
locations,  travel  through  the  same  or  different  enroutes,  and  arrive  at  different 
destinations.  Integrating  the  cycles  should  show  the  feasibility  of  a  movement  since  it 
would  take  multiple  routings,  force  them  to  share  resources,  and  show  how  the  movement 
performs.  The  improved  model  also  matches  planes  to  routes  in  order  to  optimize 
efficiency  based  on  factors  such  as  cargo  type  and  amount,  winds,  number  of  stops,  and 
aircraft  available. 

Additionally,  the  model  is  more  intuitive  for  the  user.  Although  Pektas  did  an 
admirable  job,  there  are  some  areas  of  the  model  that  are  unclear  and  cumbersome  to  use. 
The  previous  method  of  selecting  a  route  by  filling  in  multiple  boxes  is  changed  to  an 
easy  to  use  user  form  that  requests  the  locations  one  at  a  time.  The  way  new  locations  are 
input  into  the  model  is  modified  as  well;  the  process  is  easier  and  less  confusing  by 
guiding  the  user  through  the  process  using  forms. 

Another  aspect  of  improving  the  ease  of  use  of  the  model  is  modifying  the  help 
screens.  They  are  clearer  and  more  concise,  and  their  appearance  has  changed. 
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Previously,  the  user  saw  one  screen  at  a  time,  and  had  to  click  a  button  to  see  the  next 
screen.  The  help  screens  have  been  adjusted  so  that  the  user  can  scroll  through  them  like 
traditional  windows. 

The  final  improvement  is  to  resolve  compatibility  issues  to  ensure  the  model 
works  on  any  system.  Currently,  there  are  some  problems  with  the  model  that  prevent  it 
from  being  used  across  different  systems.  Since  the  model  is  intended  to  be  passed  to 
multiple  users  and  used  on  deployments,  it  must  be  easily  transferable. 

Chapter  II  of  this  thesis  includes  a  literature  review  of  modeling  and 
current  airlift  models,  Visual  Basic  and  Visual  Basic  for  Applications,  and  verification 
and  validation  techniques.  Chapter  III  presents  the  methodology  that  was  used  to 
improve  the  model  and  test  it.  Chapter  IV  explains  the  results  of  the  improvements  and 
the  verification  and  validation.  Chapter  V  presents  conclusions  and  offers  areas  for 
future  research. 
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II.  Literature  Review 


Introduction 

Mobility  modeling  is  not  a  new  field  of  research  and  study.  Planners  have  several 
resources  available  to  them  that  provide  guidance  and  procedure  when  executing  a 
mission.  This  chapter  surveys  these  resources  and  the  other  inputs  to  the  modeling 
process.  A  brief  overview  of  general  modeling  is  included.  Current  airlift  instructions 
and  models  are  discussed  to  determine  the  current  status  of  the  air  mobility  modeling 
process.  Visual  Basic  for  Applications  is  introduced  and  its  benefits  and  drawbacks  are 
presented.  Finally,  a  review  of  the  verification  and  validation  literature  is  provided. 

Modeling 

Modeling  is  the  act  of  developing  a  simplified  version  of  a  system  in  order  to  test 
a  particular  part  of  that  system.  In  virtually  all  cases,  modeling  is  less  expensive  and  less 
time  consuming  than  creating  the  system  in  its  complete,  actual  form.  For  instance,  it  is 
much  easier  to  test  the  aerodynamics  of  a  model  aircraft  wing  than  it  is  to  build  the  actual 
aircraft  and  test  it  (Ragsdale,  2001:2).  In  air  mobility  modeling,  it  is  much  less  time 
consuming  to  test  different  airlift  options  by  using  a  computer  model  that  accurately 
reflects  the  important  aspects  of  an  airlift  mission  instead  of  flying  the  different  options 
with  actual  aircraft  and  crews. 

Mathematical  models  can  be  divided  into  three  categories:  descriptive,  predictive, 
and  prescriptive.  Descriptive  models  are  used  when  the  relationship  of  the  variables  is 
known,  but  the  values  of  the  variables  are  unknown.  Descriptive  models  use  tools  such 
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as  simulation  and  program  evaluation  and  review  technique  (PERT).  Predictive  models 
are  used  when  the  values  of  the  independent  variables  are  known,  but  the  relationship 
between  the  variables  is  unknown.  Predictive  models  use  regression,  time  series,  and 
discriminant  analysis.  Prescriptive  models  are  used  when  both  the  relationship  between 
the  variables  and  the  values  of  the  variables  are  known.  They  are  used  in  linear 
programming,  network  flow  programming,  integer  programming,  and  nonlinear 
programming  (Ragsdale,  2001:8). 

Models  can  also  be  classified  over  time,  rate  of  change,  and  variation.  Static 
models  do  not  change  over  time,  whereas  dynamic  models  do.  Continuous  models  allow 
variables  to  change  continuously  over  a  range  of  values;  discrete  model  variables  can 
only  take  on  particular  values  and  cannot  fall  anywhere  outside  those  given  values. 
Deterministic  models  do  not  account  for  variation  and  will  give  the  same  answer  every 
time  given  the  same  inputs;  stochastic  models  take  variation  into  account  and  add  an 
element  of  randomness  to  attempt  to  model  uncertainty  in  the  system  (Kelton  and  others, 
2002:8). 

AMPCALC  is  a  prescriptive,  static,  discrete,  and  deterministic  model.  It  is 
prescriptive  because  the  relationships  between  the  variables  are  known,  as  are  the  values 
of  the  independent  variables.  For  instance,  we  know  that  total  mission  time  is  flight  time 
plus  ground  time,  flight  time  is  determined  by  speed,  distance,  and  other  factors,  and  we 
know  how  fast  a  given  airplane  flies,  how  long  it  must  stay  on  the  ground  to  be  refueled, 
loaded  or  unloaded,  and  to  meet  other  requirements.  The  model  is  static  because  it  does 
not  take  time  into  consideration.  It  will  not  give  a  different  result  when  given  the  same 
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inputs,  no  matter  how  much  time  passes.  AMPCALC  is  discrete  because  we  are  dealing 
with  objects  that  cannot  be  divided;  i.e.  we  cannot  use  half  of  a  plane.  Finally, 
AMPCALC  is  deterministic  because  variability  is  not  included  in  the  model.  Averages 
are  used  for  those  variables  that  could  vary  over  time,  such  as  wind  speeds  and  ground 
times  in  order  to  give  a  quick  picture  of  how  a  mission  could  be  executed. 

Current  Airlift  Models 

Air  Force  Pamphlet  10-1403,  Air  Mobility  Planning  Factors,  provides  broad 
planning  factors  for  air  mobility  operations.  It  allows  planners  to  “make  gross  estimates 
about  mobility  requirements  in  the  early  stages  of  the  planning  process”  (AFP  10-1403, 
1998:1).  The  document  provides  formulas,  planning  factors,  and  examples  of  different 
kinds  of  mobility  operations.  While  the  formulas  are  not  difficult  to  calculate,  gathering 
the  necessary  data  in  order  to  make  the  computations  is  very  time  consuming.  For 
instance,  a  planner  would  first  have  to  determine  what  needed  to  be  moved,  what  aircraft 
were  available  for  the  operation,  the  loading/unloading  times  for  the  aircraft,  the  distance 
to  be  traveled,  block  speed  for  that  distance,  fuel  consumption,  crew  limitations,  etc. 
While  most  of  the  information  is  available  in  the  pamphlet,  attempting  to  use  it  by  itself 
to  plan  a  major  airlift  operation  would  be  a  daunting  task. 

These  planning  factors  are  based  upon  a  single  cycle.  A  cycle  is  defined  as  an 
aircraft  movement  through  a  route  from  an  origin,  or  Aerial  Port  of  Embarkation 
(APOE),  through  enroutes  where  the  aircraft  may  receive  additional  cargo,  passengers, 
fuel,  minor  maintenance,  and  a  different  crew,  to  its  destination,  or  Aerial  Port  of 
Debarkation  (APOD),  where  it  is  unloaded  and  returns  to  its  origin.  A  cycle  may  include 
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one  or  many  aircraft  and  is  the  basis  of  several  of  the  calculations  used  to  describe  an 
airlift  movement. 

In  an  attempt  to  automate  many  of  the  necessary  calculations  associated  with  such 
an  operation,  Lt  Col  David  L.  Merrill  of  the  Command  Analysis  Group  at  Headquarters 
Air  Mobility  Command  (HQ  AMC/XPYS)  created  the  Airlift  Cycle  Analysis  Spreadsheet 
(ACAS)  in  1989.  Still  in  use  today,  ACAS  does  in  fact  make  planning  a  movement  much 
more  efficient.  However,  the  program  resides  completely  in  one  Excel  worksheet  that  is 
318  rows  x  83  columns.  This  makes  navigation  difficult,  and  although  there  are  some 
buttons  included  to  help  with  navigation,  trying  to  find  some  information  can  be  tricky. 
Another  shortcoming  of  ACAS  is  that  distances  must  be  determined  by  using  another 
program  and  then  input  into  the  spreadsheet.  Finally,  ACAS  deals  with  only  two  possible 
airlift  cycles.  In  today’s  environment,  movements  will  tend  to  be  more  complex,  so  more 
than  two  cycles  need  to  be  analyzed  in  order  to  make  the  best  decision. 

AMPCALC  was  designed  to  address  the  shortcomings  of  ACAS.  It  incorporates 
great  circle  distances  that  were  not  calculated  by  ACAS.  Great  circle  distance  is  simply 
the  shortest  distance  between  two  points  on  the  earth,  since  the  shortest  distance  between 
two  points  on  a  sphere  is  not  a  straight  line.  A  separate  spreadsheet  called  DISTCALC 
was  used  to  determine  Great  Circle  distances.  It  also  allows  for  four  airlift  cycles,  but 
currently  they  are  not  simultaneous.  AMPCALC  also  breaks  up  the  single  spreadsheet  of 
ACAS  into  multiple  sheets,  making  navigation  easier.  This  research  makes  AMPCALC 
an  even  better  product  by  enhancing  its  features. 
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The  Air  Mobility  Operations  Simulator  (AMOS)  is  a  simulation  of  all  AMC 
operations.  Currently  under  development  by  Emergent  Information  Technologies,  Inc, 
and  Air  Mobility  Command,  AMOS  is  designed  to  accurately  and  dynamically  model 
cargo,  passenger,  and  air  refueling  scenarios  on  both  the  tactical  and  strategic  levels. 
AMOS  uses  entities,  requirements,  resources,  command  and  control,  and  environments, 
to  build,  schedule,  and  execute  missions.  Written  in  C++,  AMOS  currently  operates  on 
Linux  systems,  with  Windows  functionality  in  progress  (Duhadway  and  others,  2001). 
AMOS  is  used  to  test  and  validate  AMPCALC. 

Visual  Basic  for  Applications 

Visual  Basic  is  an  object  based  programming  language  developed  by  Microsoft. 
Visual  Basic  is  a  stand  alone  product  used  to  create  Windows-based  programs  (Albright, 
2001:5).  Visual  Basic  for  Applications  (VBA),  on  the  other  hand,  is  similar  to  Visual 
Basic  but  runs  within  Microsoft  Office  and  other  software,  usually  without  the  user’s 
knowledge.  VBA  allows  the  user  to  increase  the  functionality  of  programs  by  telling  the 
program  to  execute  a  certain  procedure,  also  known  a  macro,  when  a  certain  action  is 
taken,  i.e.  a  button  is  clicked,  a  value  is  changed,  an  option  is  selected,  etc. 

VBA  is  used  in  the  model  because  it  offers  the  advantages  of  simplicity, 
commonality,  and  accessibility.  The  VBA  language,  although  not  necessarily  easy  to 
learn,  is  certainly  not  as  difficult  to  learn  or  use  as  other  programming  languages  that 
could  possibly  be  used.  Since  VBA  is  an  object  oriented  language,  items  are  easily 
manipulated  after  learning  relatively  simple  programming  rules.  VBA,  as  a  programming 
language,  is  ideal  for  the  Air  Force  (AF)  since  the  service  relies  heavily  on  Microsoft 
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products  and  Microsoft  Excel  can  be  found  on  virtually  every  AF  desktop  computer  and 
laptop.  VBA  is  accessible  in  that  the  code  can  be  made  available  to  anyone  as  long  as  it 
is  not  password  protected.  This  allows  the  code  to  be  updated  and  modified  as  necessary. 
VBA  was  also  stipulated  as  the  programming  language  of  choice  in  the  Statement  of 
Work  provided  by  AMC. 

Verification  and  Validation 

There  are  several  sources  that  discuss  the  proper  method  of  verifying  and  validating 
a  model.  Banks,  Gerstein,  and  Searles  outline  the  most  frequently  used  methods  in  their 
article  “Modeling  Processes,  Validation,  and  Verification  of  Complex  Simulations:  A 
Survey.”  They  found  five  “common  threads”  that  must  be  used  in  the  modeling  process: 
an  initialization  phase,  an  attack  plan,  creating  a  detailed  design,  testing  the  model,  and 
the  operation  and  maintenance  phase  (Banks  and  others,  1987:  13).  They  found  that  all 
of  the  processes  they  looked  at  incorporated  verification  and  validation  techniques  in 
every  phase,  especially  in  the  preliminary  ones  (Banks  and  others,  1987:  14).  They  use 
the  General  Accounting  Office’s  (GAO)  guidance  to  modeling  as  an  example:  in  the 
twelve  steps  of  the  process,  only  the  very  last  step,  “Transfer  system  to  user,”  does  not 
have  verification  and  validation  associated  with  it  (Banks  and  others,  1987:  14).  The 
same  GAO  report  also  gives  guidelines  on  the  minimum  requirements  of  evaluating  the 
model.  Those  guidelines  are:  documentation,  validity  (including  theoretical  validity,  data 
validity,  and  operational  validity),  computer  model  verification,  maintainability 
(including  updating  and  reviewing),  and  usability  (when  and  how  the  model  should  be 
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used)  (Banks  and  others,  1987:  15).  Banks  also  presents  R.  E.  Shannon’s  three  questions 
that  must  be  answered  while  evaluating  the  model: 

1.  Does  the  model  adequately  represent  the  real  world  system? 

2.  Is  the  model  generated  behavioral  data  characteristic  of  the  real  system 
behavioral  data? 

3.  Does  the  simulation  model  user  have  confidence  in  the  model’s  results?  (Banks 
and  others,  1987:  17). 

These  principles  are  put  into  practice  during  actual  model  evaluation,  as  evidenced 
by  Zaragoza  and  Hogan  and  their  work  on  the  Integrated  Exposure  Biokinetic  (IEUBK) 
model  for  the  U.S.  Environmental  Protection  Agency.  They  outline  their  verification  and 
validation  strategy  as: 

a.  Does  the  model  represent  the  mechanisms  of  the  modeled  system,  and  are 
those  mechanisms  sufficiently  understood? 

b.  How  extensive  are  the  data  used  to  create  the  model? 

c.  Are  the  relationships  translated  into  code  correctly  and  free  of  errors? 

d.  Can  the  model  be  compared  with  actual  data  and  how  reasonable  are  the 
results  from  the  model  in  comparison  to  the  real  world  behavior?  (Zaragoza  and  Hogan, 
1998:  1552) 

Balci  adds  that  there  are  basically  two  ways  of  verifying  and  validating  a  model: 
subjectively  and  statistically.  The  method  ultimately  used  depends  on  whether  the  data 
are  completely  observable,  partially  observable,  or  completely  unobservable  (or 
nonexistent)  (Balci,  1987:  21).  He  claims  that  modeling  is  an  art  and  is  also  situation 
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dependent  (Balci,  1987:  21).  Therefore,  by  designing  the  methodology  using  these 
principles,  the  conclusions  should  be  in  agreement  with  the  majority  of  the  concepts  of 
the  verification  and  validation  fields. 

With  a  basic  understanding  of  modeling  practices  and  current  airlift  models,  one 
can  see  the  usefulness  of  increasing  the  functionality  of  AMPCALC.  Using  Visual  Basic 
for  Applications  is  an  appropriate  programming  language  choice  in  this  case  because 
VBA  is  widely  recognized  and  is  available  Air  Force  and  DoD  wide.  Performing  the 
verification  and  validation  is  essential  in  order  to  ensure  the  model  accurately  reflects  the 
system  it  is  attempting  to  model.  Chapter  III  explains  the  approach  used  in  making 
improvements  to  the  model  and  conducting  the  verification  and  validation  of  the  model. 

It  guides  the  reader  through  the  methodology  of  testing  the  model  and  provides  the  basis 
for  interpreting  the  results. 
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III.  Methodology 


Chapter  Overview 

This  chapter  explains  the  theory  and  procedures  used  to  make  improvements  and 
test  the  validity  of  AMPCALC.  The  approach  taken  to  correctly  integrate  the  airlift 
cycles  is  discussed  and  expanded  upon  in  the  first  section.  The  steps  for  verification  and 
validation  are  outlined  in  the  second  section. 

Modeling  Methodology 

The  primary  purpose  of  AMPCALC  is  to  give  planners  a  quick,  general  idea 
about  how  an  airlift  movement  will  perform;  initial  questions  are  “Is  it  feasible?”  and 
“How  long  will  it  take?”  AMPCALC  attempts  to  answer  these  questions  as  quickly  and 
accurately  as  possible.  However,  we  must  acknowledge  that  a  deterministic  model  has 
limitations  and  can  only  do  so  much.  This  is  especially  true  considering  the  fact  that 
airlift  movements  can  be  very  complex.  The  most  basic  mission  is  relatively  simple  to 
model.  Air  Force  Pamphlet  (AFP)  10-1403,  Air  Mobility  Planning  Factors,  describes  the 
equations  necessary  to  “make  gross  estimates  about  mobility  requirements  in  the  early 
stages  of  the  planning  process”  (Department  of  the  Air  Force,  1998).  These  equations, 
which  are  used  in  some  form  in  the  model  and  are  taken  from  the  ACAS  User  Manual, 
include  but  are  not  limited  to  the  following  equations: 

Note:  parenthesis  ( )  indicate  units;  brackets  [  ]  partition  mathematical  operations; 
A  slash  /  reflects  division  and  an  X  multiplication;  +  and  -  are  used  for  addition  and 
subtraction,  respectively. 

1.  Number  of  Cargo  Missions  =  Cargo  Requirement  (tons)  /  Average  Payload  (tons) 
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2.  Number  of  Passenger  (Pax)  Missions  =  [Total  Pax  -  Pax  on  Cargo  Missions]  /  Pax 
Capability  per  Pax  Mission 

NOTE:  Pax  on  Cargo  Missions  =  Number  of  Pax  seats  available  on  each  cargo 
mission  X  Number  of  Cargo  Missions 

3.  Total  Missions  Required  =  Number  of  Cargo  Missions  +  Number  of  Pax  Missions 

4.  Round  Trip  Flying  Time  (RTFT)  =  [Feg  Distl  /  Block  Speedl  ]+  [Feg  Dist2  /  Block 
Speed2  ]+...  +  [Feg  Distn  /  Block  Speedn  ]  for  all  n  legs  of  the  cycle  from  onload  to 
offload  to  onload. 

5.  Average  Block  Speed  =  Round  Trip  Distance  (nm)  /  RTFT  (hrs) 

6.  Total  Ground  Time  (TGT)  =  Onload  Time  +  [Enroute  Time  X  #  of  Enroute  Stops  in 
Cycle]  +  Offload  Time 

7.  Cycle  Time  =  Round  Trip  Flying  Time  (RTFT)  +  Total  Ground  Time  (TGT) 

8a.  Station  Interval  (hrs)  =  Station  Ground  Time  /  Station  Capability 

NOTE:  Station  Capability  should  be  expressed  in  terms  of  numbers  of  aircraft  a 
station  is  capable  of  handling  during  the  planned  ground  time  at  that  station.  Capability 
may  be  limited  by  ramp  space,  maintenance,  civil  engineering,  transportation  (MHE), 
refueling,  net  weight  restrictions,  base  facilities,  or  command  and  control.  The  Station 
Capability  will  be  the  most  restrictive  of  these  factors  and  should  be  calculated  for  the 
station  in  the  cycle  with  the  most  prohibitive  capability. 

8b.  Aircraft  Allocation  Interval  (hrs)  =  Cycle  Time  /  PAA  Allocation  (#  of  Acft) 

8c.  Flying  Hour  Capability  Interval  (hrs)  =  [RTFTX24]  /  [Objective  Ute  Rate  X  PAA 
Allocation] 

8d.  Stage  Crew  Interval  (hrs)  =  [Crew  Rest  Period  -  Acft  Ground  Time]  /  Number  of 
Stage  Crews 

NOTE:  This  formula  is  used  only  when  the  Number  of  Stage  Crews  available  is 
known,  or  a  station's  ability  to  support  crews  is  limited  and  the  number  of  crews  which 
can  be  supported  is  known. 

9.  Flow  Interval  (hrs)  =  maximum  of  [8a,  8b,  8c,  8d] 

10.  Closure  (days)  =  {[Missions  Required  -  l]X[Flow  Interval]  +  One  Way  Enroute 
Time  (onload  to  destination)}  /  24  (Merrill,  1989). 
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As  more  planes,  cargo,  passengers,  and  locations  are  added,  the  computation  of 
the  above  equations  becomes  more  complex.  AMPCALC  currently  models  four  separate 
airlift  cycles.  In  order  to  integrate  these  cycles  to  properly  reflect  a  more  likely  modern- 
day  scenario,  significant  changes  have  to  be  made.  Integrating  the  cycles  gives  a  more 
realistic  view  of  an  airlift  operation.  In  most  cases,  in  order  to  wage  war  or  support 
humanitarian  activities  around  the  world,  there  are  cargo  and  passengers  at  different 
locations  that  must  share  the  same  number  of  aircraft  and  use  the  same  infrastructure. 
Separate  airlift  cycles  assume  that  there  is  nothing  else  going  on  in  the  airlift  system.  The 
system  includes  all  other  airplanes,  crews,  and  locations  that  have  missions  and  day  to 
day  operations  in  addition  to  those  being  modeled  by  AMPCALC.  While  separate  cycles 
still  provide  an  estimate  of  how  a  movement  might  look,  a  more  accurate  picture  is 
illustrated  when  the  cycles  are  integrated. 

The  first  step  in  integrating  the  cycles  is  establishing  a  common  pool  of  aircraft. 

A  new  interface  is  designed  for  the  user  to  input  the  number  of  aircraft  available  across 
all  cycles.  The  routing,  cargo,  and  passenger  information  is  input  for  each  cycle.  In  most 
cases,  the  cycles  could  differ  enough  such  that  maintaining  their  individual  cycle 
information  is  necessary.  For  instance,  cycle  1  would  have  a  round  trip  flying  time 
(RTFT)  that  is  different  than  cycle  2’s  which  is  different  than  cycle  3’s,  etc.  Since  cycle 
time  is  driven  by  RTFT  and  total  ground  time  (TGT),  which  also  vary  by  cycle,  these 
numbers  must  be  computed  separately  as  well.  Since  the  majority  of  formulas  depend  on 
results  from  previous  equations,  total  closure  is  found  by  taking  the  maximum  closure 
times  of  the  individual  cycles.  In  order  to  ensure  the  aircraft  are  properly  distributed 


14 


between  the  cycles,  an  optimization  program  is  developed  and  solved  using  Excel  Solver. 
The  objective  cell  is  set  to  minimize  the  maximum  of  the  four  closure  times  of  the  four 
cycles.  The  problem  is  constrained  by  the  number  of  aircraft  allocated  to  the  movement. 
As  solver  changes  the  number  of  aircraft  allotted  to  a  given  cycle,  closure  times  change. 
These  times  should  converge  to  an  optimal  solution  resulting  in  the  shortest  closure  time 
for  all  four  cycles. 

This  approach  is  tested  by  running  the  model  with  a  given  number  of  aircraft  in 
one  cycle  over  a  given  route  with  a  certain  amount  of  cargo.  The  results  are  then 
compared  to  the  same  number  of  aircraft  allotted  and  tasked  to  take  the  same  amount  of 
cargo  over  the  same  route  using  all  four  cycles. 

Verification  and  Validation  Methodology 

This  is  a  quantitative  quasi-experiment,  defined  as  an  “experimental  approach  in 
which  the  research  does  not  assign  subjects  randomly  to  treatment  and  control 
conditions”  (Dooley,  2001:  349).  Data  is  collected  from  AMC  and  the  Air  Mobility 
Operations  Simulator  (AMOS)  on  the  relevant  aspects  of  the  air  mobility  mission, 
including  number  and  type  of  aircraft,  flight  times,  ground  times,  maximum  on  ground 
(MOG)  information,  etc.  The  data  are  then  compared  to  the  model  output  given  the  same 
input.  This  approach  traces  the  data  in  an  attempt  to  determine  how  well  the  model 
performs.  The  model  diagram  is  as  follows,  with  the  first  observation  being  the  inputs  to 
the  model  and  the  real  world  mission  planning  requirements.  The  intervention  is  the 
model,  and  the  last  observation  reflects  the  model  output,  which  can  then  be  compared  to 
the  real  world  and  AMOS  results. 
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Figure  1.  Verification  and  Validation  Methodology  Model 

The  verification  and  validation  procedures  used  on  the  AMPCALC  closely  mirror 
those  done  by  Zaragoza  and  Hogan  in  their  validation  and  verification  of  the  Integrated 
Exposure  Biokinetic  (IEUBK)  model  (Zaragoza  and  Hogan,  1998:  1552).  Specifically, 
these  procedures  include: 

•  Review  the  model  documentation 

•  Compare  the  equations  in  the  model  with  the  theory  behind  the  model 

•  Verify  the  Visual  Basic  for  Applications  code  for  completeness,  accuracy,  and 
efficiency 

•  Use  a  math  software  program  (such  as  MathCad)  to  validate  equations  and 
results 

•  Conduct  analysis  of  model  output  and  actual  real  world  missions  to  determine 
how  accurately  the  model  performs 

•  Consult  the  Air  Mobility  planning  experts  at  AMC  for  their  feedback  on  the 
appropriateness  and  validity  of  the  model 

•  Consult  with  experts  from  AFIT  to  validate  the  analysis 
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•  Ensure  the  model  meets  US  Air  Force  guidelines  for  model  development  and 
structure 

•  Recommend  improvements  for  ease  of  use  and  user  interface 

•  Investigate  the  appropriateness  of  adding  variability  to  the  model  (probabilistic 
modeling) 

Upon  completing  the  above  steps,  a  very  good  indication  is  available  on  how 
effective  and  accurate  the  AMPCALC  really  is,  and  how  useful  it  might  be  to  the  Air 
Mobility  Command. 
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IV.  Analysis  and  Results 


Chapter  Overview 

This  chapter  discusses  the  results  of  the  improvements  and  modifications  to  the 
model  and  the  results  of  the  verification  and  validation  of  the  model. 

Improvement  Results 

The  model  now  includes  the  option  to  integrate  the  cycles.  An  Excel  Solver 
routine  is  written  in  VBA  and  is  accessed  via  a  button  on  the  Aircraft  Allocation  sheet. 
Pressing  the  “Integrate  Cycles”  button  brings  up  a  user  form  that  allows  the  user  to  input 
the  number  of  available  cargo  and  passenger  aircraft  that  will  be  allocated  across  all  four 
cycles.  Pressing  the  “Compute  Closure”  button  at  the  bottom  of  the  user  form  closes  the 
dialog  box  and  launches  the  solver  routine,  which  allocates  the  aircraft  over  the  different 
cycles  and  routes  in  order  to  achieve  the  fastest  closure  time  for  the  system.  The  problem 
is  bound  by  ensuring  the  number  of  aircraft  allocated  does  not  exceed  the  number 
available  and  by  ensuring  the  cargo  and  passengers  that  must  be  moved  are  indeed 
transported  from  their  origin  to  their  destination.  At  this  time,  the  solver  routine  does  not 
produce  an  optimal  answer  because  the  problem,  as  formulated,  is  not  linear.  Attempts  to 
solve  as  a  nonlinear  problem  also  did  not  succeed;  in  those  instances,  Solver  encountered 
an  error  in  the  problem  and  could  not  find  a  solution.  Solver  should  be  able  to  handle 
this  optimization.  There  are  88  variables,  or  changing  cells,  which  is  well  under  its  200 
decision  variable  limit.  Also,  there  are  only  30  constraints,  which  is  also  well  under 
Solver’s  limit  of  100  constraints  (Frontline  Systems,  2000:  9). 
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A  convenient  byproduct  of  optimizing  closure  times  is  that  planes  are  assigned  to 
routes  that  are  better  able  to  take  advantage  of  a  particular  aircraft  type  based  on 
properties  such  as  cargo  type  and  amount,  winds,  distance,  block  speed,  and  number  of 
stops.  Unfortunately,  since  the  optimization  itself  does  not  work  at  this  time,  this  feature 
is  also  not  functioning.  However,  upon  successful  implementation  of  the  Solver 
optimization,  the  air  mobility  planner  will  have  a  better  idea  about  where  to  position 
assets  to  maximize  efficiency. 

The  new  Routing  screen  has  a  “Change  Routing”  button  above  each  of  the  cycles. 
Clicking  on  this  button  brings  up  a  user  form  that  allows  the  user  to  create  or  change  the 
current  routing  of  a  given  cycle.  Pressing  the  button  above  the  respective  cycle  column 
will  default  to  changing  that  column’s  entries,  but  the  user  is  able  to  choose  which  cycle 
to  modify  by  using  the  option  buttons  at  the  top  of  the  form.  The  user  also  has  the  option 
to  completely  clear  the  current  routing  and  start  from  scratch  by  pressing  the  “Clear 
Routing”  button  on  the  left  side  of  the  form.  In  the  middle  of  the  form,  a  VBA  combo 
box  is  used  to  select  a  location,  which  can  be  found  by  using  either  the  name  or  ICAO 
code,  which  is  determined  by  another  set  of  option  buttons  to  the  right  of  the  combo  box. 
Once  a  location  has  been  chosen,  the  user  must  assign  the  location  to  be  either  the  origin, 
enroute,  or  destination  by  pressing  the  respective  buttons  located  below  the  combo  box. 
As  enroute  locations  are  added,  additional  options  appear  next  to  the  enroute  location.  A 
“Change”  button  appears  next  to  each  entry  that  allows  the  user  to  modify  that  location. 
Pressing  the  “Change”  button  clears  that  position  in  the  routing  and  allows  the  user  to 
choose  another  location  in  its  place.  Any  empty  locations  in  the  enroute  column  are 
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automatically  filled  in  with  the  previous  entry’s  location,  indicating  that  no  travel  is 
taking  place  between  those  two  points.  A  textbox  appears  for  the  user  to  input  average 
wind  conditions  to  that  location.  A  combo  box  appears  for  the  user  to  select  the  stop  type 
at  that  location.  After  updating  the  routing,  combo  boxes  are  used  to  determine  what 
kind  of  stop  is  assigned  to  the  location.  Previously,  the  user  had  to  manually  enter  a 
number  between  one  and  five  in  order  for  the  model  to  determine  if  the  location  was  an 
onload,  a  navigational  waypoint,  an  enroute,  an  offload,  an  engine  running  offload 
(ERO),  or  another  non  standard  stop  type.  The  user  now  simply  selects  what  kind  of  stop 
takes  place  at  a  location  by  choosing  that  option  from  the  combo  box  to  the  right  of  the 
location.  Similarly,  Yes/No  option  buttons  appear  to  the  right  of  the  combo  boxes  to 
indicate  whether  or  not  crews  are  staged  at  the  enroute  location.  The  user  can  now  see  at 
a  glance  what  kind  of  stop  occurs  and  if  crews  are  staged  at  a  given  location.  When  the 
route  is  complete,  the  user  presses  the  “Update  Routing”  button.  This  button  fills  in  any 
blanks  left  by  the  user  and  places  the  values  in  their  respective  positions  on  the  routing 
input  page.  This  approach  results  in  the  ability  to  make  modifications  to  routes  more 
quickly  and  easily  and  should  be  much  easier  for  the  user  to  understand.  Buttons  have 
also  been  added  that  allow  the  user  to  simply  press  “Yes”  or  “No”  to  the  query  of 
whether  the  routes  are  used  for  one  way  traffic  or  if  traffic  is  moving  in  both  directions. 
Pressing  “Yes”  tells  the  model  that  the  locations  must  deal  with  airlift  coming  from  both 
directions,  whereas  pressing  “No”  tells  the  model  the  airlift  is  only  going  in  one  direction 
through  the  route. 

Figure  2  illustrates  the  new  routing  form. 
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Figure  2.  The  New  Routing  User  Form 

The  model  also  has  a  dialog  box  that  allows  the  user  to  add  new  locations  to  the 
ICAO  list.  Pressing  the  “Add  New  Location”  button  on  either  the  Routing  or  Distance 
Calculation  sheet  displays  the  “Add  New  Location”  window,  which  allows  the  user  to 
input  the  necessary  items  associated  with  a  location.  These  items  include  the  name, 
ICAO,  region,  latitude,  and  longitude  of  a  new  location.  The  user  selects  North/South  for 
latitude  and  EastAVest  for  longitude  with  option  buttons.  The  region  selection  is  a  combo 
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box  that  lists  the  regions  currently  included  in  the  ICAO  list.  The  user  can  simply  choose 
from  the  list  or  input  a  new  region  identifier.  Pressing  the  “Add  Location”  button  adds 
the  information  to  the  ICAO  list,  formats  the  entries  properly,  and  sorts  the  list  by  ICAO 
and  by  location  name.  Figure  3  illustrates  the  add  location  dialog  box. 


Figure  3.  The  Add  New  Location  User  Form 
The  information  screens  designed  to  give  users  background  and  instructions  on 
each  of  the  sheets  have  been  improved.  Users  should  feel  more  comfortable  with  the 
more  traditional  windows  dialog  box  with  scroll  bars  when  appropriate  and  an  “OK” 
button  at  the  bottom  of  the  screen  that  hides  the  information  box  when  pressed.  In 
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addition,  the  information  screens  have  been  edited  for  readability  and  incorporate 
changes  that  have  been  made. 

Buttons  have  been  added  throughout  the  model  in  lieu  of  forcing  the  user  to  input 
some  kind  of  code  in  order  to  get  the  model  to  perform  in  a  particular  way.  Examples 
include  “On”  and  “Off’  buttons  for  the  Mission  Capable  rate  factor  on  the  Aircraft 
Standard  Planning  Inputs  Factors  sheet,  “AMMP”  and  “DS”  buttons  on  the  Aircraft 
Ground  Times  Input  sheet  which  tells  the  model  to  use  either  Air  Mobility  Master  Plan  or 
Desert  Storm  ground  times,  and  “No  Limit”  and  “Limit”  buttons  on  the  Aircrew  Call-up 
and  Limits  sheet,  which  determines  whether  or  not  crews  are  a  limiting  factor  in  an  airlift 
movement.  All  of  these  improvements  make  the  model  easier  to  use  by  showing  the  user 
what  can  and  cannot  be  changed  and  allows  for  easier  analysis  of  the  effects  on  closure  of 
the  different  inputs. 

Since  the  model  is  intended  to  be  passed  to  multiple  users  and  used  on 
deployments,  it  must  be  easily  transferable.  Improvements  in  this  area  are  continuing  as 
the  model  is  subjected  to  various  different  machines  with  different  versions  of  Windows, 
Excel,  VBA,  and  Excel  Solver.  Thus  far,  most  compatibility  issues  have  been  solved  by 
declaring  variables  in  the  different  subroutines  of  the  VBA  code.  Previously,  variables 
were  not  defined  prior  to  their  use  and  would  cause  the  VBA  code  to  produce  an  error 
when  it  encountered  a  variable  with  which  it  was  not  familiar.  The  model  was  tested  on 
several  different  systems  in  and  around  the  AFIT  community  and  has  met  with  success 
thus  far.  In  addition,  a  group  of  AFIT  students  was  tasked  to  test  the  model  in  order  to 
find  compatibility  issues.  All  issues  raised  by  the  group  have  been  solved. 
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Verification  and  Validation  Results 


In  order  to  verify  and  validate  AMPCALC,  the  following  questions  proposed  in 
the  literature  review  and  the  methodology  are  answered  below. 

1.  Does  the  model  adequately  represent  the  real  world  system?  In  many  respects, 
the  model  does  adequately  represent  the  real  world  system.  Many  factors  that  affect  an 
airlift  cycle  are  incorporated  into  the  model.  Mission  capable  rates,  crew  limits,  MOG 
considerations,  number  of  aircraft  available,  actual  carrying  capacities  of  cargo  and 
passengers,  different  block  speeds  based  on  distance  and  those  based  on  standard 
planning  factors  are  all  included.  However,  there  are  some  limitations;  as  a  deterministic 
model,  it  does  not  take  the  variability  that  exists  in  any  real  world  system  into  account.  It 
also  allows  the  user  to  fly  anywhere  in  the  world  when  real  world  constraints  may  not 
permit  this.  The  output  of  the  model  is  only  as  good  as  the  inputs,  and  the  better  the 
inputs,  the  more  realistic  the  outputs  will  look. 

2.  Is  the  model  generated  behavioral  data  characteristic  of  the  real  system 
behavioral  data?  The  data  generated  by  the  model  are  characteristic  of  the  real  system 
behavioral  data.  When  dealing  with  airlift  movements,  ground  times,  block  speeds,  and 
closure  times  are  used  to  determine  how  effective  a  movement  will  be.  The  model’s 
ability  to  use  the  planning  factors  that  are  input  to  the  model  accurately  produce  these 
desired  results. 

3.  Does  the  simulation  model  user  have  confidence  in  the  model’s  results?  The 

model  currently  produces  reasonable  responses  when  given  realistic  input.  The  formulas 
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used  in  the  model  have  been  validated  and  the  user  should  have  confidence  that  the 
model  works  as  intended. 

4.  Can  the  model  be  compared  with  actual  data  and  how  reasonable  are  the  results 
from  the  model  in  comparison  to  the  real  world  behavior?  The  model  results  can  and 
have  been  compared  with  actual  data  from  the  Air  Mobility  Operations  Simulator 
(AMOS)  and  are  acceptable.  As  long  as  the  user  provides  appropriate  input  to  the  model, 
the  results  will  generally  be  an  83%  solution.  The  results  of  actual  comparisons  are 
included  below. 

5.  Does  the  model  represent  the  mechanisms  of  the  modeled  system,  and  are  those 
mechanisms  sufficiently  understood?  The  model  accounts  for  the  majority  of  factors 
that  influence  an  airlift  movement,  including  number  and  type  of  aircraft,  cargo  and 
passenger  limitations  of  the  aircraft,  loading  and  ground  times,  adjusted  flying  times 
based  on  distance  and  airspeed,  maximum  on  ground  limitations,  crew  limits,  and 
mission  capable  rates.  The  model  does  not,  however,  take  into  account  the  possible 
variability  that  can  occur  over  the  time  the  movement  is  taking  place.  Many  of  the 
factors  listed  above  may  change.  The  number  of  aircraft  may  be  reduced  due  to  higher 
priority  missions.  Loading  and  ground  times  are  based  on  averages,  which  may  or  may 
not  hold  true  throughout  the  duration  of  the  movement.  Winds  that  will  affect  airspeed 
and  closure  time  may  change.  Maximum  on  ground  numbers  may  not  be  valid 
throughout  the  duration  as  the  model  does  not  account  for  all  possible  traffic  that  passes 
through  a  particular  location.  Mission  capable  rates  are  subject  to  change  as  well. 
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6.  How  extensive  are  the  data  used  to  create  the  model?  The  data  used  to  create  the 


model  was  obtained  from  several  sources,  including  Air  Force  Pamphlet  10-1403,  Air 
Mobility  Planning  Factors  and  the  Airlift  Cycle  Analysis  Spreadsheet.  These  sources 
provided  the  necessary  information  to  accurately  model  an  airlift  movement,  to  include 
equations  to  properly  model  airspeed,  closure  time,  flow  interval  times,  and  several  other 
interactions  that  are  used  in  the  model.  AFP  10-1403  provided  the  planning  factors  that 
are  used  for  ground  times,  loading/unloading  times,  servicing  times,  crew  limits,  and 
mission  capable  rates. 

7.  Are  the  relationships  translated  into  code  correctly  and  free  of  errors?  One 

benefit  of  using  Excel  is  that  the  equations  are  easy  to  set  up  within  the  model.  VBA  also 
allows  additional  control  over  the  model  and  permits  the  user  to  use  the  equations  easily 
and  more  efficiently  because  they  are  embedded  in  the  worksheets.  VBA  also  allows  the 
model  to  make  difficult  functions,  such  as  calculating  great  circle  distances,  transparent. 
In  this  case,  the  equations  necessary  to  determine  the  distance  between  two  points  were 
put  into  a  VBA  function  routine,  which  is  then  called  by  name  from  the  model  using  the 
latitude  and  longitude  of  the  two  locations. 

The  methodology  introduced  the  following  list  of  actions  to  be  completed  in  order 
to  insure  validity.  The  results  are  listed  below. 

1.  Review  the  model  documentation.  The  model’s  documentation  is  included  within 
the  model  in  the  form  of  help  screens.  These  screens  were  revised  and  updated  for  both 
clarity  and  accuracy.  They  give  an  accurate  and  informative  picture  of  the  capabilities  of 
the  model. 
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2.  Compare  the  equations  in  the  model  with  the  theory  behind  the  model.  The 

equations  used  in  the  model  match  those  given  in  AFP  10-1403  and  ACAS.  These 
equations  have  been  validated  for  accuracy. 

3.  Verify  the  Visual  Basic  for  Applications  code  for  completeness,  accuracy,  and 
efficiency.  Several  changes  were  made  to  the  VBA  code  in  the  interest  of  completeness 
and  efficiency.  One  instance  involved  replacing  over  20  pages  of  code  that  looped 
through  the  location  list  in  order  to  find  a  match  to  a  given  location  in  order  to  find  the 
location’s  latitude  and  longitude.  There  were  several  shortcomings  with  the  previous 
version  of  the  program.  First,  it  took  an  inordinate  amount  of  time  for  the  program  to 
loop  through  the  2100+  locations  in  the  model.  Secondly,  there  was  a  loop  set  up  for 
each  location,  meaning  there  were  44  instances  of  the  same  code.  Finally,  the  loops  did 
not  allow  for  locations  to  be  added  to  the  model.  Eliminating  the  redundant  code  and 
replacing  it  by  using  Excel’s  built-in  VLOOKUP  function  in  the  cells  where  the 
information  was  needed  neatly  solved  this  issue. 

A  second  instance  where  the  code  was  modified  dealt  with  the  Great  Circle 
Distance  calculation.  The  first  version  of  the  model  used  a  separate  subroutine  each  time 
a  distance  had  to  be  calculated  between  two  points.  The  actual  code  to  perform  the 
calculation  is  approximately  half  a  page  long.  Since  there  are  four  cycles,  there  were 
over  40  repetitions  of  this  code.  Several  pages  of  code  were  eliminated  by  creating  a 
function  subroutine  that  can  be  called  from  both  the  distance  calculation  sheet  and  the 
routing  sheet.  The  function  uses  the  latitude  and  longitude  of  the  locations  in  order  to 
find  the  distance.  Another  benefit  of  this  approach  is  that  the  function  is  available  in  the 
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Excel  menus  as  a  user  defined  function  that  can  be  easily  found  and  exported  for  other 
uses. 

4.  Use  a  math  software  program  to  validate  equations  and  results.  MathCAD  200  li 
Professional  was  used  to  replicate  the  equations  used  in  the  model  and  validated  the 
results  of  the  equations.  All  results  were  within  3%  of  the  model’s  output.  The 
MathCAD  templates  used  can  be  found  in  Appendix  1.  The  great  circle  distance 
calculation  was  also  compared  with  a  similar  type  calculator  found  on  the  internet  at 
http://www.gb3pi.org.uk/great.html.  While  the  calculator  on  the  website  produces 
answers  in  miles  and  AMPCALC  produces  answers  in  nautical  miles,  conversion  factors 
were  used  to  find  common  units  for  the  purpose  of  comparison.  The  distance  between 
40°  1”  north  latitude,  74°34”  west  longitude  and  50°26”  north  latitude,  7°36”  east 
longitude  was  calculated.  As  Table  1  shows,  the  percent  difference  between  the  two 
methods  is  less  than  0.1%. 


Table  1.  Comparison  Results  between  AMPCALC  and  gb3pi  Distance  Calculators 


Method 

Distance 

Conversion  factor 

Distance 

%  Difference 

AMPCALC 

3348  nm 

1.853  km/nm 

6203.844  km 

gb3pi 

3853  mi 

1 .609  km/mi 

6199.477  km 

.07 

5.  Conduct  analysis  of  model  output  and  actual  real  world  mission  results  to 
determine  how  accurately  the  model  performs.  AMPCALC  results  were  compared 
with  the  ALMTest  data  in  AMOS.  Tests  were  performed  on  AMOS  to  ensure  the  level  of 
fidelity  necessary  for  comparison.  The  model  was  run  using  1,  5,  10,  and  20  repetitions 
with  different  random  seeds.  As  the  number  of  repetitions  increased,  the  95%  confidence 
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intervals  around  the  mean  decreased  as  expected,  and  led  to  the  conclusion  that  more  than 
one  repetition  would  not  be  necessary  in  order  to  maintain  the  desired  level  of  accuracy. 
Table  2  presents  this  information. 


Table  2.  Means  and  Confidence  Intervals  for  Multiple  AMOS  Repetitions 


Number  of  Repetitions 

Mean  Closure 

95%  Cl  Lower 

95%  Cl  Upper 

1 

38 

n/a 

n/a 

5 

37.8 

37.24 

38.36 

10 

37.9 

37.67 

38.13 

20 

37.8 

37.61 

37.99 

AMPCALC  was  compared  to  AMOS  using  four  different  scenarios.  All  of  the 
scenarios  are  part  of  the  AFMTest  that  is  included  with  and  used  to  test  AMOS.  Each 
scenario  uses  a  piece  of  the  overall  movement  required  by  the  AFMTest  scenario.  In 
each  case,  the  cycle  under  investigation  was  modified  to  include  only  those  aircraft  and 
requirements  that  were  to  be  moved  from  one  APOE  to  one  APOD.  This  required 
modifying  the  input  files  in  AMOS,  particularly  the  AirUnit  and  Requirements  files.  The 
AirUnit  file  lists  all  of  the  aircraft  that  are  available  to  the  simulation.  The  file  was 
modified  to  include  only  those  aircraft  that  were  allocated  to  the  cycle,  and  the  rest  of  the 
aircraft  were  given  a  zero  balance.  The  Requirements  file  lists  all  of  the  cargo  and 
passenger  requirements  that  need  to  be  moved  from  an  APOE  to  an  APOD.  All 
requirements  that  were  not  associated  with  the  APOD/APOE  pair  under  investigation 
were  deleted.  These  modifications  allowed  for  a  better  comparison  between  AMOS  and 
AMPCALC. 
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The  AFMTest  is  a  hypothetical,  unclassified  scenario  that  takes  place  in 


southwest  Asia.  The  APOD  for  the  majority  of  cargo  and  passengers  is  Kuwait  (OKBK). 
Therefore,  three  of  the  four  scenarios  use  OKBK  as  the  APOD  in  order  to  have  enough 
throughput  to  warrant  investigation.  The  APOEs  for  those  scenarios  are  Dover  Air  Force 
Base,  Delaware  (KDOV),  Robert  Gray  Army  Air  Field,  Fort  Hood,  Texas  (KGRK),  and 
Jiyanklis  New,  Egypt  (HE8X).  The  fourth  scenario  uses  Miramar,  Marine  Corps  Air 
Station,  California  (KNKX)  as  the  APOE  and  Shaikh  Isa,  Bahrain  (OBBS),  as  the  APOD. 
The  results  of  these  comparisons  are  shown  in  Table  3. 


Table  3.  AMPCALC  and  AMOS  Scenario  Comparison  Results 


Routing 

Aircraft 

Requirements 

(stons) 

MOG 

AMPCALC 

Closure 

(days) 

AMOS 

Closure 

(days) 

Dover 

Barajas  (Madrid) 

Kuwait 

19C-5A 
20  C-17 
20WBC 

15  WBP 

0  Outsize 

87  Oversize 
4,426  Bulk 

469  Pax 

15  at  OKBK 

4.76 

4.11 

Robert  Gray  AAF 
Barajas 

Kuwait 

10C-5A 
20  C-17 
10WBC 

10  WBP 

2,456  Outsize 
2,802  Oversize 
321  Bulk 

8,185  Pax 

15  at  OKBK 

11.04 

8.15 

Logan 

Frankfurt 

Jiyanklis  New  (Egypt) 
Kuwait 

5  WBC 

20  WBP 

1  Outsize 

5  Oversize 

132  Bulk 

10,226  Pax 

3  at  HE8X 

4.81 

3.95 

Miramar 

Dover 

Moron 

Shaikh  Isa  (Bahrain) 

40  C-5A 

293  Outsize 

1406  Oversize 
184  Bulk 

1,347  Pax 

6  at  OBBS 

6.78 

6.06 

As  the  table  shows,  the  differences  between  AMPCALC  closure  times  and  AMOS 
closure  times  are  relatively  small.  Percent  differences  were  calculated  and  are  listed  in 
Table  4. 
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Table  4.  Percent  Difference  and  Percent  Accurate  Comparison  Results  between 

AMPCALC  and  AMOS 


Origin 

Destination 

AMPCALC 

AMOS 

%  Difference 

%  Accuracy 

KDOV 

OKB  K 

4.76 

4.11 

13.7 

86.3 

KGRK 

OKB  K 

11.04 

8.15 

26.2 

73.8 

HE8X 

OKBK 

4.81 

3.95 

17.9 

82.1 

KNKX 

OBBS 

6.78 

6.06 

10.6 

89.4 

Total 

17.1 

82.9 

As  the  table  shows,  AMPCALC  closure  time  is  within  17%  of  the  AMOS  closure 
time  on  average. 

6.  Consult  the  Air  Mobility  planning  experts  at  AMC  for  their  feedback  on  the 
appropriateness  and  validity  of  the  model.  Unfortunately,  due  to  current  world  events, 
AMC/XPY  has  not  been  able  to  devote  the  time  and  manpower  to  assist  in  this  endeavor 
as  much  as  they  would  have  liked.  However,  the  model  has  been  given  to  them  for  their 
review  and  use,  and  their  feedback  will  be  included  if  further  revisions  of  the  model  are 
accomplished. 

7.  Consult  with  experts  from  AFIT  to  validate  the  analysis.  Since  there  are  not  many 
analysis  tools  available  to  compare  the  results  of  a  simulation  with  the  results  of  a 
deterministic  model,  AFIT  faculty  and  AMC/XPY  analysts  agreed  that  the  analysis 
performed  was  accurate  and  appropriate. 

8.  Ensure  the  model  meets  US  Air  Force  guidelines  for  model  development  and 
structure.  Air  Force  Instruction  16-1001,  Verification,  Validation,  and  Accreditation 
(VV&A)  implements  Air  Force  Policy  Directive  16-10,  Modeling  and  Simulation 
Management,  and  outlines  the  responsibilities  of  agencies  developing  and  implementing 
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models  and  simulations  (M&S).  At  this  point  in  time,  the  AFI  does  not  govern  the 
development  of  AMPCALC  because  it  does  not  meet  the  criteria  of  a  major  model  and 
simulation.  Specifically,  AMPCALC  is  not  an  element  of  a  federation  of  models  and 
simulations,  its  application  does  not  involve  safety  of  life,  and  its  development  does  not 
involve  the  commitment  of  significant  DoD  resources.  Should  any  of  those  conditions 
change,  the  model  would  be  subject  to  the  provisions  of  AFI  16-1001,  and  AMC  would 
have  to  appoint  a  Verification  and  Validation  Manager  to  the  model  to  ensure  the 
requirements  of  the  AFI  are  met  (AFI  16-1001).  In  its  current  state,  AMPCALC  should 
have  little  difficulty  following  the  guidelines  should  the  need  arise. 

9.  Recommend  improvements  for  ease  of  use  and  user  interface.  A  mobility 
modeling  class  at  the  Air  Force  Institute  of  Technology  was  given  the  model  and 
instructions  to  provide  feedback  on  ease  of  use  and  other  issues.  Future  feedback  will  be 
incorporated  if  improvements  on  the  model  continue  to  be  made. 

10.  Investigate  the  appropriateness  of  adding  variability  to  the  model 
(probabilistic  modeling).  At  this  point,  introducing  variability  to  this  model  would 
probably  be  counter-productive.  The  main  objective  of  the  model  is  to  give  planners  a 
quick  idea  about  how  an  airlift  movement  will  perform.  Introducing  variability  may  add 
additional,  unnecessary  complexity  to  the  model  that  the  casual  user  would  not 
appreciate.  Introducing  variability  also  means  that  several  repetitions  would  have  to  be 
performed  before  a  reasonable  answer  can  be  reached.  This  detracts  from  the  model’s 
goal  of  being  a  quick  look  type  of  tool.  Finally,  in  light  of  the  current,  ongoing 
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development  of  AMOS,  which  has  the  sole  purpose  of  modeling  the  variability  of  the 
airlift  system,  adding  variability  to  AMPCALC  is  not  recommended  at  this  time. 

In  conclusion,  the  improvements  to  the  model  make  the  model  run  more  smoothly 
and  reliably,  allowing  for  wider  distribution  and  use  of  the  model.  The  verification  and 
validation  efforts  offer  encouraging  evidence  that  the  model  sufficiently  represents  the 
real  world  system  and  allows  planners  to  quickly  analyze  an  airlift  mission  and  arrive  at  a 
reasonable  assessment  of  how  it  will  perform.  The  next  chapter  discusses  the  limitations 
and  conclusions  of  this  endeavor. 
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V.  Conclusions  and  Recommendations 


In  the  process  of  improving  and  validating  the  model,  several  limitations  were 
discovered.  The  first  and  most  significant  limitation  was  that  AMC/XPY  was  not  as 
available  as  the  researcher  would  have  hoped.  Unfortunately,  due  to  unforeseen  world 
events,  issues  of  greater  importance  required  their  attention.  Ultimately  this  meant  that 
feedback  or  comparison  data  was  not  available  to  run  the  model  against,  and  therefore  the 
results  are  based  on  comparisons  to  theoretical  and  simulated  data.  Another  dilemma 
encountered  was  that  some  of  the  data  that  possibly  could  have  been  used  for  comparison 
was  classified.  In  order  to  keep  this  thesis  unclassified,  that  data  was  not  provided  or 
used. 

Another  limitation  existed  with  the  Air  Mobility  Operations  Simulator  (AMOS), 
in  that  there  were  several  problems  getting  it  installed  correctly,  and  once  installed,  was 
difficult  to  manipulate.  Additionally,  the  installation  delays  limited  its  availability. 

AMOS  is  continuously  being  updated,  and  not  having  the  latest  version  impeded  the 
progress  of  the  comparisons.  One  particularly  troubling  problem  was  the  inability  to 
modify  the  data  flat  files  that  serve  as  inputs  to  the  simulation.  Comparisons  were  made 
using  pieces  of  the  very  large,  complex  deployment.  The  Reports  Tool  of  AMOS  was 
also  having  problems  by  not  including  the  names  of  the  Aerial  Ports  of  Embarkation 
(APOE)  in  the  reports  it  generated,  making  it  exceedingly  difficult  to  determine  from 
where  the  cargo  was  coming. 

The  model  itself  has  its  own  limitations.  There  are  many  factors  involved  in  an 
airlift  mission  that  a  model  such  as  this  simply  cannot  incorporate.  For  instance,  the 
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model  does  not  take  limited  operating  hours,  special  operating  conditions,  quiet  hours, 
and  the  like  into  consideration.  It  limits  the  entire  cycle  to  the  minimum  MOG  of  one 
location,  even  though  other  bases  would  not  be  constrained  in  such  a  manner  in  a  real 
world  situation.  It  also  assumes  that  all  of  the  cargo  and  passengers  are  available  to  move 
at  time  zero.  This  is  not  necessarily  a  realistic  assumption.  Of  course,  these  drawbacks 
can  also  be  considered  strengths:  if  the  model  included  every  factor  that  affects  an  airlift 
cycle,  it  would  not  be  as  simple  to  operate  and  would  take  much  longer  to  produce  a 
result. 

Unfortunately,  there  were  improvements  that  the  researcher  set  out  to  do  that 
could  not  be  accomplished.  Integrating  the  airlift  cycles  did  not  come  to  fruition.  This 
happened  as  a  culmination  of  several  factors:  while  the  formulation  of  the  model 
appeared  to  be  linear,  Excel  Solver  indicated  that  it  was  not.  In  addition,  Excel  Solver 
was  not  working  properly  when  applied  to  the  model.  For  instance,  its  dialog  box  would 
close  when  the  “Add  Constraint”  button  was  pressed.  The  formulation  met  the  criteria 
necessary  for  Solver  to  work,  but  it  simply  would  not.  The  best  explanation  for  this 
occurrence  is  that  there  are  so  many  interactions  between  the  number  of  aircraft  allotted 
to  a  cycle  and  closure  time  that  Solver  was  losing  track  somewhere  and  would  result  in  a 
nonlinear  answer.  Furthermore,  some  Excel  functions  were  not  operating  as  they  should 
have  been.  For  instance,  whenever  the  user  tries  to  use  the  “Save  As”  command  from  the 
“File”  drop  down  menu,  Excel  would  experience  an  error  and  close.  Also,  when  trying  to 
manipulate  certain  parts  of  the  model,  particularly  the  location  data,  Excel  would 
experience  an  error  and  close. 
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Finally,  there  was  significant  difficulty  associated  with  following  on  with 
someone  else’s  code,  even  in  the  relatively  simple  language  of  Visual  Basic  for 
Applications.  Differing  experience  levels  with  programming  and  different  logic  paths 
proved  to  be  noteworthy  impediments  to  producing  the  best  possible  code.  Programming 
is  sometimes  considered  an  art,  and  it  would  be  almost  unheard  of  for  one  artist  to  pick 
up  and  finish  another’s  work  once  started. 

Despite  these  limitations,  however,  some  progress  was  made.  Even  with  the 
differences  in  programming  experience  and  logic,  some  advances  did  result  in  significant 
model  improvement.  Several  pages  of  code  were  eliminated  and  the  model  was 
streamlined.  Compatibility  issues  were  addressed  and  resolved.  The  model  is  easier  to 
use  and  more  intuitive  for  the  user  that  does  not  have  daily  exposure  to  this  kind  of 
material  due  to  the  inclusion  of  better  information  screens,  user  forms,  and  selection 
buttons.  The  model  provides  a  good,  quick  look  at  how  an  airlift  movement  might 
perform,  which  will  allow  airlift  planners  a  starting  point  before  committing  resources  to 
an  endeavor  that  probably  will  not  work.  The  verification  and  validation  shows  the 
model  is  an  appropriate  approximation  of  what  really  happens.  All  things  considered,  the 
AMPCALC  can  and  should  be  a  valuable  tool  for  mobility  planners. 

As  with  all  endeavors,  however,  there  is  still  room  for  improvement.  Areas  for 
further  study  include  readdressing  the  cycle  integration  issues,  performing  more 
comparisons  with  real  and  simulated  data,  and  improving  the  code  to  make  it  more 
efficient.  These  avenues  are  happily  offered  to  those  who  might  follow  in  my  footsteps. 
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Appendix  1.  MathCAD  Equation  Verification 


1.  Number  of  Cargo  Missions  =  Cargo  Requirement  (tons)  /  Average  Payload  (tons) 


2.  Number  of  Passenger  (Pax)  Missions  =  [Total  Pax  -  Pax  on  Cargo  Missions]  /  Pax 
Capability  per  Pax  Mission 

NOTE:  Pax  on  Cargo  Missions  =  Number  of  Pax  seats  available  on  each  cargo 
mission  X  Number  of  Cargo  Missions 


3.  Total  Missions  Required  =  Number  of  Cargo  Missions  +  Number  of  Pax  Missions 


4.  Round  Trip  Flying  Time  (RTFT)  =  [Feg  Distl  /  Block  Speedl  ]+  [Feg  Dist2  /  Block 
Speed2  ]+  ...  +  [Feg  Distn  /  Block  Speedn  ]  for  all  n  legs  of  the  cycle  from  onload  to 
offload  to  onload  (for  C-5)  (uses  corrected  distances). 


5.  Average  Block  Speed  =  Round  Trip  Distance  (nm)  /  RTFT  (hrs) 
(uses  actual  distances) 


6.  Total  Ground  Time  (TGT)  =  Onload  Time  +  [Enroute  Time  X  #  of  Enroute  Stops  in 
Cycle]  +  Offload  Time 


7.  Cycle  Time  =  Round  Trip  Flying  Time  (RTFT)  +  Total  Ground  Time  (TGT) 
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8a.  Station  Interval  (hrs)  =  Station  Ground  Time  /  Station  Capability 


8b.  Aircraft  Allocation  Interval  (hrs)  =  Cycle  Time  /  PAA  Allocation  (#  of  Acft) 


8c.  Flying  Hour  Capability  Interval  (hrs)  =  [RTFTX24]  /  [Objective  Ute  Rate  X  PAA 
Allocation] 


8d.  Stage  Crew  Interval  (hrs)  =  [Crew  Rest  Period  -  Acft  Ground  Time]  /  Number  of 
Stage  Crews 


9.  Flow  Interval  (hrs)  =  maximum  of  [8a,  8b,  8c,  8d] 


10.  Closure  (days)  =  { [Missions  Required  -  l]X[Flow  Interval]  +  One  Way  Enroute 
Time  (onload  to  destination) }  /  24 
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